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Abstract 

We propose almost instantaneous fixed-to-variable-length (AIFV) codes such that two (resp. K — 1) code 
trees are used if code symbols are binary (resp. K-aiy for K > 3), and source symbols are assigned to incomplete 
internal nodes in addition to leaves. Although the AIFV codes are not instantaneous codes, they are devised such 
that the decoding delay is at most two bits (resp. one code symbol) in the case of binary (resp. Jf-ary) code 
alphabet. The AIFV code can attain better average compression rate than the Huffman code at the expenses of a 
little decoding delay and a little large memory size to store multiple code trees. We also show for the binary and 
ternary AIFV codes that the optimal AIFV code can be obtained by solving 0-1 integer programming problems. 

Index Terms 

AIFV code, Huffman code, FV code, code tree, Kraft inequality, Integer programming 

I. Introduction 

Lossless source codes are classified into fixed-to-variable-length (FV) codes and variable-to-fixed-length (VF) 
codes, which can be represented by code trees and parse trees, respectively. It is well known that the Huffman 
coding Ul and Tunstall coding ||2l can attain the best compression rate in FV codes and VF codes, respectively, 
for stationary memoryless sources if a single code tree or a single parse tree is used. But, Yamamoto and Yokoo 
El showed that the AIVF (almost instantaneous variable-to-fixed length) coding can attain better compression 
rate than the Tunstall coding. An AIVF code uses | Yj — 1 parse trees for a source alphabet X and codewords are 
assigned to incomplete internal nodes in addition to leaves in each parse tree. Although instantaneous encoding 
is not possible since incomplete internal nodes are used for encoding, the AIVF code is devised such that the 
encoding delay is at most one source symbol, and hence the code is called almost instantaneous. Furthermore, 
Yoshida and Kida |j4l 0 showed that any AIVF code can be encoded and decoded by a single virtual multiple 
parse tree and the total number of nodes can be considerably reduced by the integration. 

In the case of FV codes, it is well known by Kraft and McMillan Theorems |I6117]E1 that any uniquely 
decodable FV code must satisfy Kraft’s inequality, and such a code can be realized by an instantaneous FV 
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code, i.e., a prefix FV code. Hence, the Huffman code, which can attain the best compression rate in the class 
of instantaneous FV codes, is also the best code in the class of uniquely decodable FV codes. However, it is 
assumed implicitly in the above argument that the best code in uniquely decodable FV codes can be constructed 
by a fixed set of codewords (in other words, a single fixed code tree) for stationary memoryless sources. But, 
this assumption is not correct generally. Actually, Yamamoto and Wei ||9l showed that we can devise more 
efficient FV codes than Huffman codes if multiple code trees can be used in the same way as the AIVF codes, 
and they called such FV codes AT-ary AIFV (almost instantaneous fixed-to-variable length) codes when the size 
of code alphabet is K. The AT-ary AIFV code requires AT — 1 code trees to realize that the decoding delay is 
at most one code symbol. Hence, in the binary case with AT = 2, multiple code trees cannot be realized. To 
overcome this defect, they also proposed the binary AIFV code such that the decoding delay is at most two 
bits. Although they proposed a greedy algorithm to construct a good AIFV code for a given source in 13, it 
is complicated and the optimal AIFV code cannot always be derived. Furthermore, only a sketch is described 
for the binary AIFV codes, which are important practically, although AT-ary AIFV codes for AT > 3 are treated 
relatively in detail. 

In this paper, we refine the definition of the binary and AT-ary AIFV codes. The binary (resp. AT-ary for 
AT > 3) AIFV code uses two (resp. AT — 1) code trees, in which source symbols are assigned to incomplete 
internal nodes in addition to leaves. Although the AIFV codes are not instantaneous codes, they are devised 
such that the decoding delay is at most two bits (resp. one code symbol) in the binary (resp. AT-ary) case. 
Furthermore, for the binary and ternary AIFV codes, we give an algorithm based on integer programing to 
derive the optimal AIFV code for a given source. 

In Section |II] we show some simple examples of ternary AIFV codes, which can attain better compression 
rate than the ternary Huffman codes. Then, after we give the formal definition of AT-ary AIFV codes for 
AT > 3, we derive the Kraft-like inequality for the AIFV code trees. Binary AIFV codes are treated in Section 
m Furthermore, we show in Section EYl that the optimal AIFV codes can be derived by solving 0-1 integer 
programming problems for the binary and ternary AIFV codes. Finally, the compression rates of the AIFV 
codes are compared numerically with the Huffman codes for several source distributions in Section |V] 

II. AT-ary AIFV codes for AT > 3. 

A. Examples of ternary AIFV codes 

We first consider a simple ternary FV code which encodes a source symbol x € X = {a, b, c, d, e} to a 
codeword in y* = {0,1,2}*. If the source distribution is uniform, i.e., P{x) = 1/5 for all x G X, then the 
entropy of this source is H^{X) — log 3 5 sa 1.465. The code tree of the Huffman code is given by Fig. [T]for 
this source, and the average code length Lh of the Huffman code is Lh = 1-6. 

Next we consider a ternary AIFV code given by Fig. |2] which satisfies the following properties. 

Definition 1 (Ternary AIFV codes): 

(A) A ternary AIFV code consists of two code trees Tq and Ti. 

(B) Each complete internal node has three children connected by code symbols ‘O’, ‘1’, and ‘2’, and each 
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incomplete internal node has only one child connected by code symbol ‘0’ q 

(C) The root of Ti must have two children connected by code symbols ‘ 1 ’ and ‘2’. 

(D) Source symbols are assigned to incomplete internal nodes in addition to leaves. But no source symbols 
are assigned to complete internal nodes. 

The AIFV code encodes a source sequence X 1 X 2 X 3 • • • as follows. 

Procedure 1 (Encoding of ternary AIFV codes): 

(a) Use To to encode the initial source symbol Xi. 

(b) If Xi is encoded by a leaf (resp. an incomplete internal node), then use Tq (resp. Ti) to encode the next 
source symbol x^+i. 

When To given by Fig. |2]is used, the codewords of a,b,c,d,e are 0, 1, 2, 10, 20, respectively. But, they are 
1, 10, 20, 21, 22, respectively, when Ti is used. For instance, source sequence ‘abac’ is encoded to ‘0.1.1.20’ 
and source sequence ‘cdebac’ is encoded to ‘2.21.20.1.1.20’, where dots ‘.’ are inserted for the sake of human 
readability, but they are not necessary in the actual codeword sequences. 

In the decoding of a codeword sequence y = yiy 2 y 3 ■ ■ ■ € y*, code trees Tq and Ti are used in the same 
way as the encoding. 

Procedure 2 (Decoding of ternary AIFV codes): 

(a) Use To to decode the initial source symbol Xi from y. 

(b) Trace y as long as possible from the root in the current code tree. Then, output the source symbol assigned 
to the reached incomplete internal node or leaf. 

(c) Remove the traced prefix of y, and if the reached node is a leaf (resp. an incomplete internal node), then 
use To (resp. Ti) to decode the next source symbol. 

For instance, if y = 10020, then the decoded sequence is dae because ‘10’, ‘O’, and ‘20’ correspond to leaves 
d, a, and e, respectively, in Tq. But, if y = 1120, b is decoded from ‘1’ in To first because there is no path with 
‘11 • • •’ in To. Then, the current code tree transfers to Ti because ‘1’ corresponds to the incomplete internal 
node of b in To. By removing ‘1’ from y, we have y = 120. Next, a is decoded from ‘1’ in Ti because there 
is no path with ‘12 • • • ’ in Ti, and the current code tree keeps Ti because ‘1’ corresponds to the incomplete 

* For simplicity, we say “a node has a child connected by code symbol ‘j’ ” if the child is connected to the node by a branch with 
code symbol "j’. 



Fig. 1. The Ternary Huffman code for X = {a, b, c, d, e}. 
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To Ti 




Fig. 2. A ternary AIFV code for X = {a, b, c, d, e}. 


internal node of a in Ti. Finally, c is decoded from ‘20’ in Ti. Note that when a source symbol assigned to 
a leaf is decoded, the decoding is instantaneous. On the other hand, the decoding is not instantaneous when a 
source symbol assigned to an incomplete internal node is decoded. But the decoding delay is only one code 
symbol even in such cases. 

We now evaluate the average code length of the ternary AIFV code given by Fig. |2l Let Lq and Li be 

the average code length of Tq and Ti, respectively. Then, we can easily show from Fig. |2] that Lq = 1.4 and 

Li = 1.8. The transition probability of code trees are given by 

Q{To\To) = Pxib) + Px{c)=0A, (1) 

Q(To|ri) = Pxib) + Px{c) + Pxid) + Px{e) = 0.8, (2) 

and the stationary probabilities of Tq and Ti are given by Q{To) = 2/3 and Q{Ti) = 1/3, respectively. Hence, 
the average code length of the ternary AIFV code is given by 

Laifv = -^Lq + -Li = — « 1.533, (3) 

which is shorter than the average code length of the Huffman code Lh = 1-6. 

Now we explain the reason why the AIFV code can beat the Huffman code. Since incomplete internal nodes 
are used in addition to leaves for encoding in Tq, Lq = 1.4 smaller than the source entropy Hq[X) ss 1.465 
can be realized. On the other hand, Li = 1.8 is larger than Lh = 1.6 because the root of Ti has only two 
children. But, from Q{Tq) > Q{Ti), Laifv is smaller than Lh in the above example. 

If IAI is even, the loss of the ternary Huffman code becomes larger because the Huffman code tree must have 
an incomplete node. Consider the case that X = {a, b, c, d} and Px{x) = 1/4 for all x € X. Then the Huffman 
and AIFV code trees are given by Fig. [3 and Fig. |4] respectively. In this case, the entropy of this source is 
Hq{X) = loggd ss 1.262, and the average code length is given by Lh = 1-5 and Laifv = 4/3 ~ 1.333. 

It is well known that if we construct the Huffman code for X^ as shown in Fig. |5] the average code 
length per source symbol L^ can be improved compared with Lh- In the case of Px{x) = 1/4, we have 
= 43/32 Ri 1.344 < Lh = 1-5. But, the Huffman code for X^ has demerits such that the size of the code 
tree increases to roughly \X\^, and the encoding and decoding delay becomes long as \X\ becomes large. 

On the other hand, by concatenating Ti to incomplete nodes of Tq and Ti in Fig. |4] we obtain a code tree 
shown by Fig. |3 Hence, the AIFV code can realize a flexible code tree for X* by using only two code trees 
To and Ti. We note that the total size of AIFV code trees is roughly 2|A’|, and Laifv = 4/3 « 1.333 is better 
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Fig. 3. The ternary Huffman code for X = {a, 6, c, d}. 


To Ti 




Fig. 4. A ternary AIFV code for X = {a, b, c, d}. 


than = 43/32 ss 1.344. Furthermore, the encoding delay is zero and the decoding delay is at most one 
code symbol in the case of AIFV codes. 

We note from Dehnition [1] that the root of Ti must have two children. But, the root of Tq can become 
an incomplete nodej^ For instance, consider a source such that Px{a) = 0.8, Px{V) = 0.1, and Px{c) = 
Px{d) = 0.05. In this case, the entropy is given by P[^{X) ss 0.6448, and the Huffman code of this source 
is given by Fig. |3] which attains Lh{X) = 1.1. On the other hand, the ternary AIFV code shown in Fig. [T] 
attains Laifv = Q{To)Lq + Q{Ti)Li = (5/9)0.4+ (4/9)1.2 k. 0.7556 for this source. Note that Lh cannot 
become shorter than 1 in any case while Laifv can become shorter than 1 by assigning the source symbol 
X with Px{x) ^ 0.5 to the root of Tq. For instance, if we use the AIFV code shown in Fig. |2l a source 
sequence aabaaacd is encoded to ‘A.1.00.A.1.A.21.02’, where A represents the null codeword and dots are 
not necessary in the actual codeword sequence. Hence the codeword sequence y is given by y = 10012102. 
Although the code length of a in Tq is zero, we can decode xi = a from the codeword sequence y because 
we begin the decoding with Tq, and there is no rath with ‘1 • ■ • ’ in Tq, which means that xi is a. Similarly 
we can decode ‘aabaaacd" from y = 10012102. □ 

B. K-ary AIFV codes 

In this subsection, we generalize ternary AIFV codes to iF-ary AIFV codes with code alphabet y = 
{0,1,2,-- - ,77- 1} for AT > 3. 


^The idea of assigning a source symbol to the root of Tq was suggested by Prof. M. Nishiara at the presentation of m 
^ Refer Remark [2] for how to detect the end of a source sequence. 
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Fig. 5. A ternary Huffman code tree for 



Fig. 6. The concatenated code tree of AIFV code. 

Definition 2 (K-ary AIFV code): 3 

(A) A AT-ary AIFV code consists of AT — 1 code trees, Tq, Ti, • ■ •, Tk- 2 - 

(B) Each complete internal node has K children connected by code symbols ‘O’, ‘1’, ‘AT — 1’. Every 
incomplete internal node has at least one and at most AT — 2 children connected by code symbols ‘O’, ‘1’, 
■ • •, ‘A'c — 1’, where Kc is the number of the children. 

(C) The root of is called complete if it has K — k children. Eor 0 < fc < AT — 2, the root of Tk has K — k 
children connected by code symbols ‘/c’, ‘/c + 1’, • ■ • ‘K — V if the root is complete. For 0 < k < K — 3, 
the root of Tk can become incomplete, and the incomplete root of Tk must have at least one and at most 
K — k — 2 children connected by ‘fc’, ‘fc + 1’, • • •, ‘ATc — 1’, where Kc — fc is the number of the children of 
the incomplete root. We regard the incomplete root of Tk with Kc — k children as an incomplete internal 
node with Kc children. 

(D) Source symbols are assigned to incomplete internal nodes in addition to leaves. But no source symbols 
are assigned to complete internal nodes. 

A AT-ary AIFV code can encode a source sequence xiX 2 X 3 --- and decode a codeword sequence y = 

2 / 12 / 22/3 • ■ • in the same way as ternary AIFV codes. 


“^This definition is slightly different from 0 Definition 1] because the root of code tree Tk, 0 < k < K — 3, can become incomplete 


in this paper although it must be complete in (9] Dehnition 1]. 
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To Ti 




Fig. 7. A ternary AIFV code. 


Procedure 3 (Encoding of K-ary AIFV codes): 

(a) Use To to encode the initial source symbol Xi. 

(b) When Xi is encoded by a leaf (resp. an incomplete internal node with j children), then use Tq (resp. Tj) 
to encode the next source symbol Xi+i. 

Procedure 4 (Decoding of K-ary AIFV codes): 

(a) Use To to decode the initial source symbol Xi from y. 

(b) Trace y as long as possible from the root in the current code tree. Then, output the source symbol assigned 
to the reached incomplete internal node or leaf. 

(c) Remove the traced prefix of y, and if the reached node is a leaf (resp. an incomplete internal node with 
j children), then use Tq (resp. Tj) to decode the next source symbol. 

As an example, an AIFV code is shown in Fig. [8]for the case of AT = 4 and X = {a, b, c, d, e, f, g, h, i, j}. 
When source sequence ‘ abacgcebhd" is encoded by this AIFV code, the codeword sequence and the transition 
of code trees are given in Table |I] Note that when source symbol Xi is encoded (or decoded) at a node with 
j children, then Xi+i is encoded (or decoded) by Tj. Furthermore we can easily check that every Xi can be 
uniquely decoded. For instance, X 2 — b k encoded to codeword ‘1’ at incomplete internal node 6 in Tq. In 
this case, xq is encoded in Ti because the incomplete internal node b has one child in Tq. This means that 
the codeword of xq does not begin with ‘O’. In the decoding, we obtain y = 113130 • • • after the decoding of 
xi = a in To and removing decoded codeword ‘0’ from y. Then we can decode X 2 = b because there is no 
path with y = 11 ■ • • in Tg but the path ‘1’ corresponds to node 6 in Tq. 

Another example of 4-ary AIFV code trees for X = {a, b, c, d, e, f, g, h} is shown in Fig. |9] in which the 
roots of Tq and Ti are incomplete. The codeword sequence for ‘badbacgaec’ is shown in Table In] where ‘A’ 
represents the null codeword. Note that the incomplete root of T^ with Kc — k children is regarded as an 
incomplete internal node with K,. children as explained in Definition |3(C). Hence, for instance, node X 2 = a 
is the incomplete root with one child in Ti, and it is regarded as an incomplete internal node with 2 children. 
Hence Xq is encoded (or decoded) in T 2 . In the decoding, we can decode xi = b from y = 03210 • ■ • in Tq 
because there is no path with y = 03 • • • in Tg, but path ‘0’ corresponds to node b. In the decoding of X 2 , we 
have y = 3210 • • • in Ti. But, there is no path which begins with ‘3’. Hence we obtain X 2 = a because ‘no 
path’ means the root in Ti. 
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To Ti T2 





Fig. 8. An example of 4-ary AIFV code trees. 


TABLE I 

An example of codeword sequence for 4-ary AIFV CODE. 


i 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

Current code tree 

To 

To 

Ti 

T2 

Ti 

To 

Ta 

Ta 

To 

Ti 

Source symbol Xi 

a 

b 

a 

c 

9 

c 

e 

b 

b 

d 

Codeword 

0 

1 

1 

31 

30 

2 

33 

30 

1 

11 

Number of children of node Xi 

0 

1 

2 

1 

0 

2 

2 

0 

1 

0 


Since some source symbols are assigned to incomplete internal nodes, the AIFV code is not an instantaneous 
code. But since the following theorem holds, this code is almost instantaneously decodable. 

Theorem 1: The AT-ary AIFV codes defined in Definition |2] and Procedures [3] and |4] are uniquely decodable, 
and the maximum decoding delay is at most one code symbol. 

Proof: From Procedure |3}(b) and Procesure |ll-(c), both encoding and decoding have the same transition of 
code trees. Hence, each source symbol Xi is decoded in the same code tree used in the encoding. It is clear 
from Procedure |4]-(b) that if Xi is encoded at a leaf in Tk, then Xi is uniquely decodable. If Xi is encoded at 
an incomplete internal node with j-children in Tk, then the children are connected by one of code symbols 
{O,!,-- - ,j — 1} from the incomplete internal node. On the other hand, Xi+i is encoded in Tj, in which any 
path begins with one of code symbols {j, j + 1 • • • , AT — 1}. Hence the node reached in Procesure |4l-(b) is the 
same incomplete internal node used in the encoding. 

It is obvious that when Xi is encoded at a leaf, then it can be decoded instantaneously. But, when Xi is 
encoded at an incomplete internal node in Tk, we must read one more code symbol to judge whether the 
incomplete internal node corresponds to the longest path in Tk- Hence the maximum decoding delay is at most 
one code symbol. 


Q.E.D. 

Remark 1: If there are no incomplete internal nodes with j children in all code trees, we can delete the 
code tree Tk-j- Furthermore, if we use only the incomplete internal nodes with j children for a fixed j. 
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To 



Ti T2 




Fig. 9. An example of 4-ary AIFV code trees with incomplete roots. 


TABLE II 

An example of codeword sequence for 4-ary AIFV CODE WITH INCOMPLETE ROOTS. 


i 

I 

2 

3 

4 

5 

6 

7 

8 

9 

10 

Current code tree 

To 

Ti 

T2 

Ti 

To 

T2 

To 

Ti 

T2 

Ti 

Source symbol Xi 

b 

a 

d 

b 

a 

c 

9 

a 

e 

c 

Codeword 

0 

A 

32 

to 

A 

31 

13 

A 

33 

31 

Number of children of node Xi 

1 

I 

I 

0 

2 

0 

I 

I 

2 

0 


1 < J < T — 2, then the code trees can be reduced to two code trees Tq and Tx-j even for the case of iiT > 3. 
Such restriction worsens the compression rate of the K-ary AIFV codes. But, the construction of code trees 
becomes easy as shown in Section IV. 

Remark 2: In the decoding described in Procedures |2] and |4] we assumed that the end of codeword sequence 
can be detected by another mechanism. In the case that the end cannot be detected and/or the null codeword 
is assigned to an incomplete root, we add a special symbol EOF to X, and we assign EOF to a leaf in each 
Tfe. By encoding EOF at the end of a source sequence, we can know the end of the decoding. The end of 
decoding can also be detected by adding the length of a source sequence encoded by e.g. Elias 5 code Eol 
into the prefix of the codeword sequence. These worsen the compression rate a little. But, the degradation is 
negligible if \X\ is not small and the length of a source sequence is sufficiently large. 

C. Kraft-like inequalities for K-ary AIFV code trees 

In this subsection, we derive lower and upper bounds of average code length Lk for code tree T^, 0 < k < 
K-2. 

Let A/q (resp. Mj ) be the set of leaves (resp. incomplete internal nodes with j children) in code tree T^, 
0 < k < K — 2, and let Ux be the incomplete internal node or leaf corresponding to a source symbol x G X. 
Eurthermore, let lk{x) be the code length of x G X in T^. 

We first consider Tq. If G then we can change the node Ux to a complete internal node by adding 
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K — j children at depth lo{x) + 1 of Tq. Hence, we have from Kraft’s inequality that 


K-2 


Y, Y {K - j)K-Vo(-)+^] = l. 


(4) 


In the case of A: > 0, since the root of Tk has K — k children, K should become [K — k) *'=(“). 

Therefore, we have 


K-2 


Y Y {K - j){K= 1. 


(5) 


Let Px{x) = {K—j){K — k) for Hx G A/^- . Then, from J2xgx Px{x) = 1 and — log^ Px{x) = 


lk{x) + \ogj^{K -k)- \ogji{K - j), we have 

Px{x) 

rx \x) lug^ 

xGX 


0<f^(Px||P;c) = ^Px(ai)log^ ^ ^ ^ 

Px[x) 

= -Hk{X) - Y Px{x) log^ Px{x) 


xga: 

K-2 


= -Hk{X)+Y Y Px{x)[lkix)+\ogii{K-k)-logii{K-j)] 


= -HxiX) 


Y Px{x)lk{s 


.XG^ 


K-2 


+ logx(A: -k)-Y logx(A: - j) 


K-2 


= -Hk{X) + Tfc + log^(i^ -k)-Y logx(^ - j), 


( 6 ) 


i=o 


where P{AfY) = J2x-n Px(x). Hence, Lk must satisfy that 


K-2 


Afc>Px(^)+ ^P(ArW)log^ 


j=o 


K-j 
K-k 


(7) 


Next we derive an upper bound of L^. If we allow that there exist leaves and/or incomplete internal nodes 
with no source symbol assigned in Tk, (0 becomes 

K-2 

Y {K - j){K<1. (8) 

X-.TIxGMP^ 

Clearly, the original Tk can attain better compression rate than such a relaxed code tree Tk- We can easily 
check that Tk can be constructed if it satisfies 0 and incomplete internal nodes can be arranged to satisfy the 
following condition. 

Condition 7.-@ Every node n G has j children. 

We now define lk{x) as 

K-f 


lk{x) = 


'togji Px(.x) +\ogji 


K-k 


< - logx Px{x) + log^ i + 1. 


(9) 


^Refer Section [IV-BI to see how this condition can be represented by equations. 
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Then, this lk{x) satisfies and Condition [T] can be satisfied by setting j appropriately for each x because 
it can always be satisfied for any x by j = 0. Hence, for appropriately selected j, we can construct Tk with 
average code length Lk satisfying that 

< Hk{X) + ^ log^ + 1. (10) 

j=0 

Note that the term log^(i4r — j)/{K — fc) in (|7]) and (fTOl i is negative if j > k although it is positive if j < k. 
Especially, in the case of Lq, the second term of (I7]l and (fTOl l is always negative. 

The global average code length Laifv is given by 

K-2 

Laifv = ^ Q{Tk)Lk, (11) 

k=0 

where Q{Tk) is the stationary probability of Tk, and Q{Tk) is determined from Q{Tj\Tk) = 0 < 

k<K — 2, 0<j<K — 2. Generally, it is difficult to evaluate the term in (ITTll given by 

K-2 K-2 . K-2 K-2 

Q(Tk) E logK = E E QiTM^Tk) log^ ( 12 ) 

k—O j—0 k—0 J—0 

But, in the case of iT = 3 or the case such that only two code trees are used for iT > 3 as described in 
RemarkIJl it holds that Q{Tk)Q{Tj\Tk) = Q(Tj)Q(Tk\Tj). Hence, in these cases, (fTSli becomes zero, and the 
following bound is obtained from O and (UOjl-linii. 

Hk{X) < Laifv < HxiX) + 1 (13) 

Unfortunately, the upper bound in (fTsT l is the same as the well known bound of the Huffman code. But, this 
fact does not mean that the performance of AIFV code with two code trees is the same as the performance of 
the Huffman code. The AIFV code trees are more flexible than the Huffman code tree. The term ‘+1’ in (|9]l 
can be decreased by selecting j appropriately for each x € X in the case of AIFV code trees. Actually, as we 
will show in Section |Vl] the AIFV codes can attain better compression rate than the Huffman codes. 

HI. Binary AIFV codes 

A. Definition of binary AIFV codes 

The iT-ary AIFV codes treated in the previous section can be constructed only for AT > 3, and the binary 
represented codewords of AT-ary AIFV codes are not so short as binary Huffman codes. But, we show in this 
section that if decoding delay is allowed at most two bits, we can construct a binary AIFV code that attains 
better compression rate than the binary Huffman code. 

We first show a simple example of a binary AIFV code in Fig. [TO] which satisfies the following properties. 
Definition 3 (Binary AIFV codes): 

(A) A binary AIFV code consists of two code trees Tq and Ti. 

(B) Each complete internal node has two children connected by code symbols ‘O’, and ‘1’. Incomplete internal 
nodes, each of which has one child, are divided into two categories, say master nodes and slave nodes. 
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Fig. 10. A binary AIFV code. 

The child of a master node must be a slave node, and the master node is connected to its grandchild by 
code symbols ‘00’. 

(C) The root of Ti must have two children connected by code symbols ‘0’ and ‘1’. The child connected by 
‘0’ is a slave node and the root cannot have a grandchild connected by code symbols ‘00’. 

(D) Source symbols are assigned to master nodes in addition to leaves. But no source symbols are assigned 
to neither complete internal nodes nor slave nodes. 

The binary AIFV code encodes a source sequence X 1 X 2 X 3 ■ • • as follows. 

Procedure 5 (Encoding of binary AIFV codes): 

(a) Use To to encode the initial source symbol Xi. 

(b) When Xi is encoded by a leaf (resp. a master node), then use Tq (resp. Ti) to encode the next source 
symbol 

If we use the binary AIFV code shown in Fig. [TO] then for instance, a source sequence ‘cbcaab’’ is encoded to 
‘11.10.11.01.0.10’, and source sequence ‘cadbca’ is encoded to ‘11.01.1100.10.11.01’, where dots ‘.’ are not 
necessary in the actual codeword sequences. 

A codeword sequence y = j/ij/ 22/3 • • • S 3^* can be decoded by using code trees Tq and Ti as follows. 
Procedure 6 (Decoding of binary AIFV codes): 

(a) Use To to decode the initial source symbol xi from y. 

(b) Trace y as long as possible from the root in the cutTent code tree. Then, output the source symbol assigned 
to the reached master node or leaf. 

(c) Let y be the path from the root to the reached master node or leaf. Then, remove y from the prefix of 
y. If the reached node is a leaf (resp. a master node), then use Tq (resp. Ti) to decode the next source 
symbol. 

For instance, from y = 11101101010, we can decode xi = c when ‘111’ is read because there is no path ‘111’ 
from the root in Tq but the master node c is reached by ‘11’. Similarly, in the case of y = 11011100101101, 
we can decode xi = c when ‘1101’ is read because there is no path ‘1101’ in Tq. We can easily check that 
‘cadbca’ can be decoded from y = 11011100101101. We note that Xi is decoded instantaneously if Xi is 
encoded by a leaf, and it is decoded with two-bit delay if Xi is encoded by a master node. Hence, the decoding 
delay of the binary AIFV codes is at most two bits. 

Now consider a source such that X = {a, b, c, d}, and Px{o) = 0.45, Px(b) = 0.3, Px{c) = 0.2, Px{d) = 
0.05. In this case, the entropy and the average code length of the binary Huffman code are given by H 2 {X) ss 
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Fig. 11. A binary AIFV code with the incomplete root in Tq. 



1.7200 and Lh = 1-8, respectively. If we use the binary AIFV code shown in Fig. [TOl the average code length 
are given by Lq = 1.65 and Li = 2.1 for Tq and Ti, respectively. Since Ti is used only just after c is encoded 
in Fig. [TOl we have Q{Ti\To) = 0.2 and Q{Tq\Ti) = 0.8 which mean that Q{To) = 0.8 and Q{Ti) = 0.2. 
Therefore, we have Laifv = 1-65 x 0.8 + 2.1 x 0.2 = 1.74, which is better than Lh = 1.8. 

Note that the root of Tq can become a master node although the root of Ti must have two children. Such an 
AIFV code is shown in Fig. [TT]for X = {a,b,c}. For instance, source sequence a;ia: 2 a :3 = aaab is encoded to 
codeword sequence ‘A.1.A.010’ by this AIFV code, which means y = 1010. We can decode a;ia; 2 a ;3 uniquely 
from y = 1010. First, we decode xi = a because there is no path with ‘1 • • • ’ in Tq. This means that xi is 
encoded at the root of Tq, and hence xi = a. Next we move to Ti, and we obtain X 2 = a from y = 1010. 
Then, we move to Tq with y = 010. Since there is no path with ‘1 • • • ’ in Tq, we decode 0:3 = a. Finally we 
move to Ti with y — 010, and we obtain X 4 = b. When Px{o) = 0.9 and Px{b) = Px{c) = 0.05, this AIFV 
code have that Q(Ti\Tq) = 0.9, Q{To\Ti) = 1, Q(To) = 10/19, Q{Ti) = 9/19, Lq = 0.3, Li = 1.2, and 
Laifv = Q{Tq)Lq + Q{Ti)Li m 0.7263. On the other hand, this source has H{X) m 0.5690 and the average 
code length of the Huffman code is Lh = 11. In the binary case, Lh cannot become shorter than one while 
Laifv can become shorter than one as shown in this example. 

B. Kraft-like inequalities for binary AIFV codes 

In the same way as Section III-CI we can derive Kraft-like inequalities for binary AIFV codes. Let Nq^'^ 
(resp. be the set of leaves (resp. master nodes) in code tree Tk, fc = 0,1. Furthermore, let Ux be the 

master node or leaf assigned a source symbol x, and let lk{x) be the code length of x € X. Note that since a 
master node has only one grandchild, the master node becomes a complete node if we add three grandchildren 
to the master node. Hence we have the following relation for Tq. 

-p ^ = 1 . (14) 

Similarly, the following relation holds for Ti because the root of Ti can have only three grandchildren. 

2 -h(x) 3 Y 2-bW = ^ (15) 

x:nxGAfQ^^ x:nxGAf^^^ 

or 

4 Y = 1. (16) 

x:nxGAfQ^^ x:nxGAfi^^ 


August 3, 2015 


DRAFT 





14 


Furthermore, the global average code length Laifv is given by 
Laifv = Q{To)Lo + Q{Ti)Li 

Then, in the same way as (|7]i, (fTOl i. and (fOl) . we can derive the following bounds. 

H 2 {X) - P(Arl°^)(2 - log 2 3) < Lo < ff 2 (X) - P(A/'|°^)(2 - log 2 3) + 1 , (18) 

ff2(X) + P(Ar^^^)(2 - log2 3) < Li < Il2(X) + P(JV^'^)(2 - log2 3) + 1, (19) 

ff2(X) < Laifv < ff2(X) + 1, ( 20 ) 

where the upper bounds of the above inequalities must satisfy the following condition. 

Condition 2: ^ Every node n G fc = 0,1, has one grandchild. 

Note that Lq can become smaller than the source entropy H 2 {X) but Li is larger than H 2 {X). Although 
the upper bound H 2 {X)+\ in (l20l l is the same as the case of Huffman codes, the term ‘+1’ can be decreased 
than the Huffman codes for individual sources because the binary AIFV code trees are more flexible than the 
Huffman code tree. 


IV. Construction of AIFV code trees based on integer programming 

In this section, we propose a construction method of AIFV code trees based on integer programming (IP) 
for AIFV codes with two code trees. Although the IP problem is generally NP hard, the IP is used to solve 
more practical problems as the hardware of computers and the software of IP solvers develop. 

Before we treat AIFV code trees, we first consider the case of binary Huffman code trees. Fet X = 
{ai,a 2 ,--- ,a|;f|}, pt = Px{<it), and dt = l{at). Then, the problem to obtain the binary Huffman code 
tree is equivalent to obtain {dt} that minimizes ^tJiPtdt under the Kraft inequality 
l-fl 

( 21 ) 

t=i 

In this case, the inequality ‘<’ in (l2Tll can be replaced with equality ‘=’ because the optimal {dt} always 
satisfies the equality in (1211 1. 

In order to formalize this optimization problem as a 0-1 IP problem, we introduce binary variables ut,d such 
that ut,d = 1 if source symbol a* is assigned to a leaf of depth d in a code tree, and ut,d = 0 otherwise. Then, 
the optimization problem can be formalized as follows. 


IP Problem 1: 

\X\ D 


minimize 

'^Ut,dPtd 

(22) 


d^l 



D 


subject to 


(23) 


d^i 



D 



Y^ut,d = l, f = l,2,...,|A'|, 

(24) 


d=l 


®Refer Section llV-AI to see how this condition can be represented by equations. 
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where D is a positive integer constant, which represents the maximum depth considered in the IP problem. 

Condition (El guarantees that each at is assigned to only one d, and dt is determined as dt = d for 
Ut^d = 1.0 must be sufficiently large. But, large D consumes computational time and memory. In many cases, 
it is sufficient that D is several times as large as log 2 \X\. 


A. IP problem for binary AIFV code trees 


In order to obtain the optimal binary AIFV code for a given probability distribution {pt}, we need to construct 
an IP problem that minimizes Laifv = Q{T[))Lq + Q{Ti)Li. However, in such IP problems, we need a lot 
of variables because we must treat two code trees at once. Furthermore, since Q{Tq)Lq and Q{Ti)Li include 
nonlinear terms, many additional variables and conditions are required to linearize nonlinear terms. Hence, 
although we can formalize an IP problem to obtain the global optimal solution, it becomes impractical or can 
treat only a small size of X. Therefore, in this subsection, we derive individual IP problems for Tq and Ti 
that can attain near-optimal Laifv, we show in Section HV-CI that the global optimal AIFV code can be 
obtained by solving the individual IP problems finite times. 

Since we can assign source symbols to master nodes in addition to leaves in the case of binary AIFV code, 
we introduce binary variables vt^d, in addition to Ut^, such that vt^d = 1 if source symbol at is assigned to 
a master node of depth d, and vt,d = 0 otherwise. Then, an IP problem to construct Tq can be formalized as 
follows. 

IP Problem 2: 


minimize 


subject to 


\X\ D 

{ut,d d + vt,d jd +C2)) 

t=l d =0 


\x\ D . . 

yz yy2 ^ (ut,d+^vt,d] = 1 , 

D 

'^{ut,d + Vt^d) = 1 , 

d=0 



D 




t=d+2 t=l 


[uty + J 



< 0 , 


(25) 

(26) 

f = l,2,...,|A|, (27) 

d = 0,l,--- ,D-2, 


(28) 


where C 2 = 2 — log 2 3 ~ 0.405. 

Furthermore, an IP problem to derive Ti is obtained by setting utp = Vtp = 0 for all t (or removing the 
case of d = 0 in dTST l- dTSl O and replacing (|2^ with the following condition: 


l-fl D 

i—1 d—1 

Condition 


“t” . '^i,d 1 — . ■ 


(29) 


comes from (fT4l i. and condition dZTl i guarantees that each at is assigned to only one of either 
leaves or master nodes. The code trees are obtained by assigning at to a leaf (resp. a master node) of depth d 
if the solution has ut^d = 1 (resp. Vt^d = !)■ 

Note that C 2 in (l25T l and Eq. (EH) are newly introduced in IP problem El compared with IP problem [T] We 
first consider why C 2 is required. 
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A leaf of depth d has weight 2~‘^ in (l26l l while a master node of depth d has weight (3/4)2“'^. Hence, 
average code lengths Lg and Li can be decreased by making many master nodes in Tq and Ti, respectively. On 
the other hand, this increases and P(A/'|^^), and hence Q{Ti) because of Q{Ti\Tq) = and 

Q(Ti\Ti) = P{J\f^^'^). Note that the global average code length is given by Laifv = QiTo)Lo + Q(Ti)Li, and 
Li is much larger than Lq because the root of Ti cannot have a grandchild with code symbols ‘00’. Therefore, 
Laifv is not always minimized even if Lq and Li are minimized individually. 

Note that if a master node is used to encode a source symbol, we must use Ti, instead of Tq, to encode 
the next source symbol. This means that master nodes have the cost L'j^jpy — Laifv compared with leaves, 
where L'j^jpy is the average code length of the case that we start the encoding with Ti instead of Tq. 

Since we derive the code trees Tq and Ti by solving separate IP problems, it is hard to embed the exact cost 
into each IP problem. But, the optimal code trees have a good property such that every child of a node has 
approximately half probability weight of its parent node. So, as an approximation of exact cost, we can use 
the cost of the ideal case such that every node has two children with equal probability weight. In this case, the 
cost is given by (72 = 2 — log 2 3 because the root of Tq can have four grandchildren while the root of Ti can 
have only three grandchildren. Therefore, cost C 2 is added for master nodes in (|25]) . 

Next we consider (l28T l. This comes from Condition shown in Section UlI-BI Each master node of depth d 
requires a slave node of depth dpi and a node or leaf of depth d + 2. Therefore, we cannot make master 
nodes of depth d if there are not sufficient number of nodes or leaves at depth d + 2. Let iV™ and be 

the number of master node of depth d and the number of nodes and leaves of depth d + 2, respectively. Then, 
TV™ is given by 
\x\ 

= (30) 

t=i 

On the other hand, we can know the number of nodes and leaves of depth d + 2 by calculating the Kraft’s 
weight at depth d + 2. Hence, N2p2 given by 

= E E 

i=d+2 t=l ' 

Furthermore, there are master nodes of depth d + 1, each of which requires one node or leaf of depth 

d + 3. Since a node or leaf of depth d + 3 has weight 2~^ at depth d + 2, we must use 2“^ out of JV^p 2 
for master nodes of depth d + 1. This means that the remaining A^^2 ~ 2“^ A^j^ nodes and leaves of depth 
d + 2 can be used for A™ master nodes of depth d. Hence, the condition (|28]) is required. 

B. IP problem for ternary AIFV code trees 

In order to obtain near-optimal ternary AIFV code, we can formalize an IP problem for ternary AIFV code 
trees in the same way as binary AIFV code trees. 
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IP Problem 3: 

1^1 D 

minimize ^ {ui,dd + Vi,d {d + C 3 )) 

d —0 


D 

subject to 

d=0 



1 


^max 

(ut.d+ft,d) = 1, 

d=0 


l-^l 

H ^t,d - 

t=l 


3^+1-^ 

i=d+l 


Zl 


1^1 

E 

t=i 


Ut,t 




< 0 , 


i = l,2,...,|A’|, 

d= ,L>- 1. 


(32) 

(33) 

(34) 

(35) 


where Ca = 1 — logg 2 a:! 0.369. 

Furthermore, an IP problem to derive Ti is obtained by setting utfi = Vtfi = 0 for all t (or removing the 
case of d = 0 in (l32T l- (l35T l') and replacing (l3?t with the following condition: 



The cost Cg for incomplete internal nodes is given by L'j^jpy — Laifv in the ideal case such that every 
child of each node has equal probability weight. Since the roots of Tq and Tg can have three and two children, 
respectively, in the ternary case, we have Cg = logg 3 — logg 2. 

Condition (l35T l is required from Condition [1] shown in Section ITl-CI and it can be derived in the same way as 
(l28l l. But, since slave nodes do not exist in the ternary case, we do not need ^vt,d+i in the first term of (l28T l. 

A new binary variable Zd is introduced in IP problem [3] compared with IP problem |2] Note that the ternary 
Huffman code has one incomplete node in the code tree when |<T| is even. Similarly a ternary AIFV code may 
have one incomplete node in Tq and/or Tg, which is not assigned any source symbol. Variable Zd represents 
the pruned leaf of such an incomplete node, = 1 if there is the pruned leaf at level d, and Zd = 0 otherwise. 

We can represent the condition (l3^ without using Zd as follows. 


D \x\ 

EE 3 

But, since the condition 


Ut,d + -Vt,d ) < 1- 


(37) 


cannot be represented without Zd, (l3^ is used rather than dJTl) . Since the pruned 
leaf must have the longest depth if it exists, we have = 1 for d = max{d : Ut^d = 1, / = 1, 2, • • • \X\} and 
zg = 0 for d ^ d in the optimal Tq and Tg. But these conditions are not explicitly included in IP problem [3] 
because the optimal code trees can be obtained without these conditions. 

Remark 3: IP problem|3]can be applied to the AT-ary AIFV codes with two code trees Tq and Tk-j explained 
in Remark [T]by modifying 2, 3, Cg and Zd in (l32T l- (l35l l as follows: 


3 K, 2 —>■ AT — j, Cq —)■ Ckj — 1 — ^ogp{K — j), 

Zd & {0, 1} —>■ Zd € {0,1, • • • , AT — 2}. 

( 7 ") 

We can also construct IP problems for general AT-ary AIFV code trees by using binary variables j to represent 
incomplete internal nodes with j children for 1 < j < AT — 2 instead of vt^d used in IP problem |3] But, the 
necessary number of variables increases and each condition described in ‘subjet to’ becomes long as AT becomes 
large. Therefore, it is hard to treat large K practically because of time and/or space complexity. 
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C. Global Optimaization 


In IP problems |2 and [2 costs C 2 and Ca are determined based on the ideal code trees such that every child 
of each node has equal probability weight. But, since the code trees Tq and Ti obtained by IP Problem |2] (or 
O do not attain the perfect balance of probability weight, they are not the optimal AIFV code trees generally. 
So, we calculate new cost C based on the obtained code trees Tq and Ti, and we derive new code trees for 
the new cost by solving again IP Problem |2] (or [2. In this section, we show that the global optimal code trees 


can be obtained by repeating this procedure. 

Let is the (m—l)-th cost and let and be the TO-th AIFV code trees obtained by solving the 

IP problem for cost is the initial cost. Furthermore, let and be the average code length 

of and respectively, and let ^g"*^ and be the transition probabilities of code trees Tg™^ and 
which are defined by = P(Afl°^ in Tg^™^) and = g(To|ri) = in 

Then, we consider the following algorithm. 

Algorithm 1: 


1. Set TO = 1 and (7*^°^ = C for given initial cost C. 

2. Obtain Tg"*^ and by solving IP problem |2] (or [2 for cost 

3. Calculate for Tg^"*^ and for 

4. Update cost as follows. 

r ("*) r (”i) 

^(m) _ - ^0 

(m) (m) 

% +9i 

5. If (7*^™^ = then exit. Otherwise, increment to and go to step 2. 


(38) 


We can use any C for the initial cost. But, if we use C 2 = 2 — log 2 3 and C 3 = 1 — logg 2 as the initial cost 
in the binary and ternary cases, respectively, Tg^^ and become near-optimal code trees. 

Theorem 2: The binary AIFV code and the ternary AIFV code obtained by Algorithm [1] are optimal. 

Proof We first prove that Algorithm [T] stops after finite iterations. First note that for the objective 

function (l25l l in IP problem |2 (or (l32T i in IP problem |2 ) can be represented as 

(39) 

Similarly, the object function for can be represented as 

L^'")+(7(™“i)(l-gJ'")). (40) 

Since (7*^"*“^^ is fixed in the IP problem used in step 2 of Algorithm [1] the minimization of (l40l) is equivalent 
to the minimization of 




On the other hand, the global average code length L^Xifv and 

, s „(™) r (™) I „("i) r ("i) 

j(m) _ Ql ^0 + % 

AIFV (rn) (m) 

% +Ti 


is given by 


(41) 


(42) 
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Since and are optimal trees that minimize (IWt and (HTt for C*-"* the following inequalities 
hold for any code trees Tg with (Lo,qo) and Ti with {Li,qi). 


lW + <Lg + C(™-i)go, 

(43) 


(44) 


Hence if we substitute Tg = Tq"* and Ti — t[^ into (|4^ and (l44l l. respectively, we have the following 
inequalities. 


_ r ("*-!) 

~ ^AIFV 

j^{m) _ ^(m-l)^(rn) ^ ^(m-1) _ 

_ jim-l) 

— ^AIFV 

If we obtain from gS]) that 

j{m) _ T{m) 

^AIFV ~ ^0 + % 

/ x(m-l) 

- ^AIFV ■ 


(45) 


(46) 


(47) 


Similarly, if we have from (l46l l that 

jim) _ j{m) 

^AIFV — 9l 

< l(-) _ C(™-1 )^(™) 

<L^Z~fv. (48) 

Therefore, if ^ we have that L^Xi^py < L^^ifv ■ Since L^^ifv ^ ^ conclude 

that L^Xifv converges as m —oo. Furthermore, since the number of code trees is finite, the convergence is 
achieved with finite m, i.e. = C"!™-!) occurs and Algorithm 1 stops after finite iterations. 

Next we prove that the obtained AIFV code trees are optimal when Algorithm 1 stops. Assume that Algorithm 
1 stops dim = m, and and are the obtained AIFV code trees that satisfy If this pair 

(jj(m) gjQ^^jjy optimal, there exists the optimal pair of code trees (Tg ,T*) with (Lg, L*, gg, g*) 

such that 


T ("*) ^ r * 

^AIFV ^ ^AIFV- 


(49) 


Then, we have for C* = (L* — Lg)/{qg + g*) that 


T* _ r* J- c*n* — T* C*n* 

^AIFV — Fg A-qg — — L, q^. 


(50) 
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Hence, if C* > we have 

j-{m) _ rim) 

^AIFV ~ ^0 + % 

— 2jq -1-0 gp 

<L* + 

<L* + C*q* 

= I^*AIFV^ 


(51) 


where the first inequality and the last equality hold from (|4^ and (fSOl l. respectively. Similarly if C* < C^'^\ 
we have 

= l(-) _ 

<Ll- 

<Ll- C*ql 


— T* 

— ^AIFV- 


(52) 


Since (ISTT i and (l52l i contradict with ( |49] ). the pair of obtained code trees {Tq\t^™’) must be globally optimal. 


Q.E.D. 


V. Performance of binary and ternary AIFV codes 


In this section, we compare numerically the performance of AIFV codes with Huffman codes. For X = 
{ai,a 2 • • ■ ,a\x\}^ we consider the following three kinds of source distributions; 


PxHa-t) = 


(53) 

(54) 

(55) 


where Ai = X]l=i ^ A 2 = normalizing constants. 

The performance of AIFV codes is compared with Huffman codes and Huffman codes for in Figs. fT2l - [T3] 
(resp. Figs. fT4l - [T6l l for the binary (resp. ternary) casj^ 

The comparison for is omitted in the binary case because the compression rate of AIFV codes is equal 
to the one of Huffman codes. The AIFV codes are derived by Algorithm [T] 

In the figures, the vertical line represents the normalized compression rate defined by Laifv/H 2 iX) and 
Lh/H 2 {X) (resp. Laifv/H?,{X) and Lh/H^{X)) for the binary (resp. ternary) case. The horizontal line 
stands for the size of source alphabet. We note from Figs. [T2UT61 that the AIFV codes can attain better 
compression rate than the Huffman codes in all cases. Furthermore, in the cases of and Px \ the binary 


^Figures 3-6 and 8 in are not correct although the algorithms sho\vn in are correct. 
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Fig. 12. Comparison between binary AIFV coding and Fluffman coding for Px'^- 



Fig. 13. Comparison between binary AIFV coding and Huffman coding for ■ 


AIFV codes can beat even the Huffman codes for and the ternary AIFV codes can attain almost the same 
compression rate as the Huffman codes for X^. 

The Huffman coding for X^ has demerits such that the size of Huffman code tree increases to roughly \X\'^, 
and the encoding and decoding delay of the first source symbol of (xi,X 2 ) € X^ becomes large as \X\ becomes 
large. On the other hand, in AIFV coding, the size of code trees is roughly 21^1 for these binary and ternary 
caselj, and encoding delay is zero and decoding delay is at most two bits (resp. one code symbol) in binary 
(resp. iF-ary for K > 3) case. Hence, from the viewpoints of coding delay and memory size, AIFV coding is 
superior to Huffman coding for X^ when \X\ is large. 

Finally we remark that if we use C = C 2 = 2 — log 2 3 (resp. C = C 3 = 1 — log 3 2) as the initial cost in 
Algorithm [1] for the binary (resp. ternary) case, is often optimal without iteration. Furthermore, even 


*In the K-sry case for K > 3, the size of AIFV code trees is roughly {K — lllA"!. 
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Fig. 14. Comparison between ternary AIFV coding and Huffman coding for P- 


(0) 



Fig. 15. Comparison between ternary AIFV coding and Huffman coding for P 


Cl) 


if L^]jfv optimal, the improvement by the iteration of Algorithm [T] is within only 0.1% compared with 

^AiFV cases of Px \ ^ ■ "^his means that if we use C = C 2 (resp. C = C 3 ) in IP 

problem (resp. IP problem |3]l, we can obtain the optimal or near-optimal AIFV codes by solving the IP 
problem for Tq and Ti only once without using Algorithm [T] 

VI. Conclusion 

In this paper, we proposed binary and AT-ary (for K > 3) AIFV coding for stationary memory less sources, 
and we showed that the optimal AIFV codes can be obtained by solving integer programing problems for 
the binary and ternary cases. Furthermore, by calculating the compression rate numerically for several source 
distributions, we clarified that the AIFV coding can beat Huffman coding. 

The following are open problems: obtain a tight upper bound of Laifv given in (fTTI) . obtain a simple 
algorithm to derive the optimal binary AIFV codes and/or the optimal AT-ary AIFV codes. 
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Fig. 16. Comparison between ternary AIFV coding and Fluffman coding for Pj 


( 2 ) 


The AIFV codes proposed in this paper are devised such that decoding delay is at most one code symbol 
(resp. two bits) in AT-ary (resp. binary) case. But, if decoding delay is allowed more than one code symbol 
(resp. two bits), it may be possible to construct non-instantaneous FV codes that can attain better compression 
rate than the AIFV codes. It is also an interesting open problem to obtain the best non-instantaneous FV codes 
for a given maximum decoding delay. 
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